Skip to content

ENH: JupyterLite integration for the documentation - #13925

Draft
natinew77-creator wants to merge 146 commits into
mne-tools:mainfrom
natinew77-creator:jupyterlite-gh-actions
Draft

ENH: JupyterLite integration for the documentation#13925
natinew77-creator wants to merge 146 commits into
mne-tools:mainfrom
natinew77-creator:jupyterlite-gh-actions

Conversation

@natinew77-creator

@natinew77-creator natinew77-creator commented May 27, 2026

Copy link
Copy Markdown
Contributor

Tracking Issue: #13929

What does this implement/fix?
This PR integrates JupyterLite into the MNE-Python documentation build, allowing users to run tutorials interactively directly in their browser without a local Python environment.

Key technical implementations:

  • Integrates jupyterlite-sphinx into the Sphinx-Gallery pipeline, automatically generating "Try in JupyterLite" buttons for tutorials and examples.
  • Injects a hidden setup cell into the generated notebooks via conf.py to automatically handle Pyodide-specific browser quirks:
    • Installs mne and pyodide-http natively via micropip.
    • Patches Pyodide networking (pyodide_http.patch_all()) so MNE's pooch downloader can successfully fetch datasets from the browser.
    • Monkey-patches mne.viz.utils.plt_show to correctly render MNE's Matplotlib figures inline within the WebAssembly environment.

Additional information

  • This represents the completion of the first major GSoC milestone.
  • CircleCI will now automatically build the JupyterLite assets and provide a live preview link in the CI checks.
  • Note on limitations: During testing, I identified two architectural edge cases for future discussion: browser RAM limitations when tutorials attempt to download massive (>1GB) datasets, and occasional PyPI vs. main branch version mismatches since JupyterLite currently pulls the stable MNE release.
A28CDACE-D867-491E-8B11-013DD1635D4A 16D95C3F-B372-47D1-AFE3-D5E10395F775 0F9F0048-D585-4B8C-BE90-1E4D34BF1EEF

@natinew77-creator

natinew77-creator commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

Hi @teonbrooks, Status Update: This PR is now ready for your review!

As the first step in my GSoC roadmap, this PR successfully introduces the core JupyterLite infrastructure to the MNE-Python documentation. Here is what has been achieved:

  • Integrated jupyterlite-sphinx to automatically build interactive JupyterLite instances for the examples/tutorials.
  • Added a GitHub Actions CI/CD workflow to build and deploy the JupyterLite site.
  • Patched MNEBrowseFigure by fixing the pyodide_plt_show argument signature to accept multiple positional arguments.
  • Fixed an extension execution race condition in doc/conf.py to ensure JupyterLite successfully bundles the generated notebooks during the Sphinx build-finished event.

I also looked deeply into the [Errno 26] Operation in progress error we hit during the 10_overview tutorial. It turns out this is a fundamental limitation with Pyodide and JupyterLite. The 10_overview tutorial uses mne.datasets.sample.data_path() to download a 1.45 GB dataset from osf.io. First, osf.io has strict CORS headers that completely block browser-based WebAssembly fetches. Second, even if we were able to bypass the CORS restrictions, downloading a 1.45 GB file directly into browser RAM via Pyodide causes the browser tab to instantly crash with an Out-Of-Memory error.

To gracefully handle this and prevent user confusion, I've written a custom pooch.Pooch.fetch interceptor in our doc/conf.py configuration. Now, whenever a JupyterLite user tries to download these massive OSF datasets natively in the browser, it intercepts the request and prints a polite error message advising them to download the dataset locally and upload it directly into the JupyterLite file browser!

For smaller datasets that are CORS-friendly, it automatically falls back to Pyodide's native pyfetch via urllib so they work seamlessly without any patching needed in the tutorial code itself.

I think this is the best architectural approach for handling the massive tutorials. I'm ready to mark this PR as complete so we can move down the GSoC checklist and start tackling xeus-python and the 3D PyVista rendering! Let me know what you think! Looking forward to your feedback!

@natinew77-creator

Copy link
Copy Markdown
Contributor Author

Quick Follow-up:
I also tracked down and fixed the bug that prevented the notebooks from loading correctly in the JupyterLite UI.

It turned out to be a race condition during the Sphinx build-finished event—jupyterlite_sphinx was executing before sphinx_gallery had finished generating the example notebooks. I've reordered the extensions in conf.py so they execute in the correct sequence, and all the notebooks are now successfully populating!

Integrates jupyterlite-sphinx into the MNE-Python doc build so every
sphinx-gallery example gets a 'Try in Browser' button backed by a
Pyodide/WebAssembly kernel.

- doc/conf.py: configure jupyterlite_sphinx; build a local MNE dev
  wheel with relaxed Pyodide constraints; copy required MNE sample-data
  subset into JupyterLite's virtual filesystem; inject a setup cell that
  installs MNE via micropip (keep_going=True bypasses version conflicts),
  mocks missing stdlib modules (lzma, multiprocessing), patches pooch to
  block large OSF downloads, and sets MNE_DATA paths
- .circleci/config.yml: ensure MNE sample data is on disk before the
  doc build so conf.py can copy it into jupyterlite_contents/
- .github/workflows/jupyterlite.yml: standalone GH Actions workflow on
  the jupyterlite-gh-actions branch that builds and uploads the site
- pyproject.toml: add jupyterlite-pyodide-kernel and jupyterlite-sphinx
  to the [doc] extras
- .gitignore: exclude jupyterlite_contents build artifacts
- mne/parallel.py: return False early in _running_in_joblib_context()
  on emscripten; joblib parallel backends are unavailable in the browser
- mne/utils/config.py: catch Exception (not just ValueError) when
  loading the MNE config JSON; Pyodide's json parser raises SyntaxError
  on a corrupt or absent config file
Tutorials and examples that call interactive Qt backends (raw.plot(),
epochs.plot(), ica.plot_sources(), etc.) or depend on large datasets not
bundled in JupyterLite will hang or error in Pyodide. Wrap them with
sys.platform guards so they are skipped when running in the browser.

Interactive Qt plots (skip on emscripten):
- tutorials/intro/10_overview.py: raw.plot(), stc.plot()
- tutorials/intro/15_inplace.py: original_raw.plot(), rereferenced_raw.plot()
- tutorials/intro/20_events_from_raw.py: raw.copy().pick().plot(), raw.plot()
- tutorials/intro/40_sensor_locations.py: mne.viz.plot_alignment()
- tutorials/evoked/40_whitened.py: raw.plot(), epochs.plot()
- examples/preprocessing/muscle_ica.py: all ica.plot_* calls

Large datasets unavailable in the browser (raise RuntimeError on emscripten):
- tutorials/io/60_ctf_bst_auditory.py: BST auditory dataset (~2.9 GB)
- tutorials/io/70_reading_eyetracking_data.py: EyeLink misc dataset
- examples/visualization/eyetracking_plot_heatmap.py: EyeLink dataset
natinew77-creator and others added 13 commits June 26, 2026 09:35
…erLite

- doc/conf.py: Fix lzma mock to use real stdlib lzma when available in
  Pyodide instead of LZMAFile=object which broke joblib's compressor
  registration
- 10_overview.py: Guard ica.plot_properties() which opens an interactive
  Qt window
- 15_inplace.py: Guard set_eeg_reference block which fails under
  Python 3.13 in Pyodide
- 20_events_from_raw.py: Guard STIM channel plot and all EEGLAB sections
  that require the unavailable testing dataset
- 40_sensor_locations.py: Guard ssvep dataset loading and sphere plot
  that require the unavailable ssvep dataset
- 50_configure_mne.py: Guard KIT test data loading whose test files are
  stripped from the Pyodide wheel
- 70_report.py: Skip Report.save() file-writing in browser, guard
  nibabel-dependent add_bem, 3D methods (add_trans/add_stc/add_forward/
  add_inverse_operator), missing ECG/events files, pandas-dependent
  make_metadata, and the HDF5 round-trip section

All intro tutorials (10, 15, 20, 30, 40, 50, 70) now run cleanly in
JupyterLite/Pyodide without errors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
set_3d_view and the other scene-level helpers read renderer.backend directly
rather than going through _get_renderer, so they were still calling into None.
It asks make_field_map to subdivide the helmet mesh through VTK, and plot_field
needs the interactive viewer, so neither blocker is reachable from the browser.
plot_alignment and plot_bem locate their surfaces by probing the filesystem
before any reader runs, so the browser has to fetch the candidates up front.
Also hides the launch badge on 14 notebooks that need the Brain widget layer
or data too large to bundle.
read_raw_nirx and read_raw_egi are handed a folder rather than a file, so the
copy leaves a manifest beside it for the browser to walk. Files past a size
limit are skipped rather than bundled, and the seven notebooks that need
fsaverage, mne_bids or a multi-hundred-MB volume model are excluded instead.
…rcle full]

The curated lite_data archive only carries the files it was published with, so
anything added since went missing from the browser bundle without failing the
build. Prefer whatever CI already restored and fall back to the archive.
The tutorials no longer branch on sys.platform: the data those blocks skipped
is served now, and reading a KIT file or a BrainVision header goes through the
setup cell like everything else. 70_report keeps too much it cannot render, so
its badge is hidden instead.
The full build reports 379 MB and 251 MB for these, so the copy step skips
them and the badge would have had nothing to load.
Its raw alone is 344 MB, 404 MB with the forward and surfaces, which is more
than those six pages are worth on every docs deploy.
@natinew77-creator

Copy link
Copy Markdown
Contributor Author

PR status comment
Update on the above.

The 31 files that showed a badge but failed are down to zero. 53 pages have the button hidden, 153 have it and their data actually loads.

The sys.platform guards are gone too, so every tutorial and example is byte-identical to upstream now and all the browser code sits in doc/conf.py.

3D now covers plot_alignment, plot_bem and source estimates through pyvista-js. Brain and plot_field still don't work, so those pages are excluded.

Dropped somato after checking with Teon, 404 MB for six pages was a lot to ship.

Most MNE readers validate a path through _check_fname before opening it, so
hooking that covers read_info, read_evokeds, read_cov and the rest at once;
read_label, read_epochs and read_raw_edf open directly and are wrapped
individually. Also serves talairach.xfm, the auditory/visual labels and the
EEGBCI runs CI provides, and installs mffpy and python-picard.
@natinew77-creator natinew77-creator changed the title ENH: Add JupyterLite CI/CD infrastructure ENH: JupyterLite integration for the MNE-Python documentation Jul 29, 2026
@natinew77-creator natinew77-creator changed the title ENH: JupyterLite integration for the MNE-Python documentation ENH: JupyterLite integration for the documentation Jul 29, 2026
… full]

dig_mri_distances goes through mne/surface.py rather than _freesurfer, so the
existing shim never fired. setup_source_space also needs surf/{lh,rh}.sphere.
An oct-6 source space meant 8196 meshes and 8196 actors in a single scene,
which ran the browser tab out of memory. Also honor glyph_resolution/height
and solid_transform, so the MRI fiducials come out 5 mm rather than a metre.
The tutorials that build a renderer themselves reach for renderer.figure,
which the pyvista backend exposes alongside scene(). Honour a fig argument
too, so plot_alignment and plot_dipole_locations composite into one scene
instead of opening a second one.
…e full]

mne.gui.coregistration places fiducials by clicking the scalp, and the vtk.js
renderer has no picker, so that one cell cannot run. Swap it for a note via
sphinx-gallery's notebook_modification_function, which only touches the
JupyterLite copies, and the other nine cells keep working.
…ircle full]

conf.py asserts sphinx_gallery_conf is serializable and sphinx rejects function
objects, so passing the hook directly broke the build before Sphinx started.
sphinx-gallery imports a dotted path instead, so move it to sphinxext.
first_notebook_cell is applied when sphinx-gallery generates a notebook, so the
piplite cell also landed in the .ipynb offered for download, where it fails on
the first line. Prepend it at copy time instead, alongside the cell notes, so
only the JupyterLite copies carry it.
Twelve of the readers only need their file fetched before MNE opens it, and
each had its own six-line copy. One wrapper plus a table of readers does the
same job; the ones that need more than a fetch keep their own shim.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants